Add Nvidia inference specification #5794
Conversation
Jan-Kazlouski-elastic
added
specification
ml
skip-backport
|
Following you can find the validation changes against the target branch for the API.
You can validate this API yourself by using the |
# Conflicts: # output/openapi/elasticsearch-openapi.json # output/openapi/elasticsearch-serverless-openapi.json # output/schema/schema.json
| }, | ||
| "dependencies": { | ||
| "@redocly/cli": "^1.34.5" | ||
| "@redocly/cli": "^1.34.6" |
There was a problem hiding this comment.
I don't think this should be getting changed here.
| "rerank", | ||
| "text_embedding", | ||
| "completion", | ||
| "chat_completion" |
There was a problem hiding this comment.
Nitpick, but could these be in alphabetical order?
| */ | ||
| model_id: string | ||
| /** | ||
| * For a `text_embedding` task, the maximum number of tokens per input before chunking occurs. |
There was a problem hiding this comment.
This should be "For a `text_embedding` task, the maximum number of tokens per input. Inputs exceeding this value are truncated prior to sending to the Nvidia API."
This is wrong almost everywhere in the docs; there's an issue describing some of the problems with max_input_tokens.
| text_embedding, | ||
| completion, | ||
| chat_completion, | ||
| rerank |
There was a problem hiding this comment.
For consistency, could these be in alphabetical order?
| */ | ||
| input_type?: NvidiaInputType | ||
| /** | ||
| * For a `text_embedding` task, the method to handle inputs longer than the maximum token length. |
There was a problem hiding this comment.
To help differentiate this from max_input_tokens it might be better to word it like "the method used by the Nvidia model to handle inputs longer than..."
| /** | ||
| * The URL of the Nvidia model endpoint. | ||
| */ |
There was a problem hiding this comment.
Would it be helpful to include the default URLs for each task type if url isn't specified?
| text_embedding, | ||
| chat_completion, | ||
| completion, | ||
| rerank |
There was a problem hiding this comment.
For consistency, could these be in alphabetical order?
This PR adds changes to specification caused by elastic/elasticsearch#132388
Additional actions